Skip to content

Enhance JSON schema validation with contextual suggestions for fixes#1045

Merged
pelikhan merged 5 commits into
mainfrom
copilot/fix-85ff7b33-5062-4293-b6b3-2250a1dda502
Sep 26, 2025
Merged

Enhance JSON schema validation with contextual suggestions for fixes#1045
pelikhan merged 5 commits into
mainfrom
copilot/fix-85ff7b33-5062-4293-b6b3-2250a1dda502

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Sep 26, 2025

This PR significantly improves the developer experience when encountering JSON schema validation errors by providing contextual suggestions for fixes.

Problem

Previously, schema validation errors provided only generic messages that didn't help users understand what valid alternatives were available:

Unknown property: invalid_property
got string, want integer

This made it difficult for users to quickly fix validation errors, especially when dealing with typos or unfamiliar properties in complex nested configurations.

Solution

The validation system now analyzes the JSON schema to provide intelligent suggestions:

Field Name Suggestions

When users provide invalid property names, the system extracts valid field names from the schema and suggests close matches:

Unknown property: timeout_minuts. Did you mean one of: timeout_minutes, ...
Unknown property: unknown_permission. Did you mean one of: actions, attestations, checks, contents, deployments, discussions, id-token, issues, ...

Type Error Examples

When the wrong data type is provided, the system generates example JSON showing the correct format:

got string, want integer. Expected format: 42

Multiple Error Handling

The system can handle multiple validation errors and provide comprehensive suggestions:

Unknown properties: permision, timeout_minuts. Valid fields are: permissions, on, description, timeout_minutes, safe-outputs

Key Features

  • Smart similarity matching: Uses heuristic scoring to find the closest field name matches
  • Schema-aware suggestions: Automatically extracts valid properties from complex JSON schemas
  • Nested object support: Properly handles oneOf schemas and deeply nested configurations
  • Example JSON generation: Creates realistic examples based on schema definitions
  • Performance optimized: Early returns and efficient similarity calculations
  • Backward compatible: All existing error handling preserved

Technical Implementation

The implementation adds several new functions to pkg/parser/schema.go:

  • generateSchemaBasedSuggestions(): Main entry point for generating contextual suggestions
  • extractAcceptedFieldsFromSchema(): Navigates JSON schema to find valid properties
  • generateFieldSuggestions(): Creates human-readable field suggestions with similarity matching
  • generateExampleJSONForPath(): Generates example JSON objects from schema definitions
  • resolveSchemaWithOneOf(): Handles complex oneOf schema patterns correctly

All magic numbers have been extracted as named constants, and comprehensive test coverage ensures reliability across different validation scenarios.

Impact

This change transforms validation errors from cryptic messages into actionable guidance, significantly reducing the time developers spend debugging configuration issues in agentic workflows.

Original prompt

Update the rendering of json schema validation by providing suggestions for fixes based on the schema.

  • if a field name is unexpected, suggest accept fields
  • if the wrong object type is provided, show example of correct JSON (generated on the fly from the schema)

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits September 26, 2025 14:10
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Generate field suggestions for unexpected properties
- Show example JSON for type errors
- Handle oneOf schemas correctly
- Add comprehensive tests for suggestion functionality

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ions

- Extract hardcoded values as named constants for better maintainability
- Add early return for empty invalidProps parameter
- Optimize similarity scoring with early exit for poor matches
- All tests pass and functionality remains intact

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Update the rendering of json schema validation by providing suggestions for fixes based on the schema. - if a field name is unexpected, suggest accept fields - if the wrong object type is provided, show example of correct JSON (generated on the fly fr... Enhance JSON schema validation with contextual suggestions for fixes Sep 26, 2025
Copilot AI requested a review from pelikhan September 26, 2025 14:30
@pelikhan pelikhan marked this pull request as ready for review September 26, 2025 14:35
@pelikhan pelikhan merged commit 79690c6 into main Sep 26, 2025
13 of 14 checks passed
@pelikhan pelikhan deleted the copilot/fix-85ff7b33-5062-4293-b6b3-2250a1dda502 branch September 26, 2025 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants